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with goals for more students not just to 
enter college, but to finish, and to raise the 
proportion of Americans with postsecondary credentials, states and the federal government are 
implementing or considering policies that would place the emphasis on completion. 


With a goal to reach the national average in higher education attainment by 2025, Tennessee is 
at the vanguard of this movement and has aligned its budget with its ambitions. It is the first state to 
base most state appropriations to colleges and universities on outcomes, with emphasis on progress 
toward and completion of degrees. Initial results are promising, but still preliminary. 


What is the policy? 


In January 2010, Tennessee passed the Complete College Tennessee Act (CCTA), which 
mandated a change to outcomes-based funding. In fall 2010, after developing outcomes measures in 
consultation with institutions and governing boards, the Tennessee Higher Education Commission 
(THEC) submitted the FY 2011-12 Budget Request, using an outcomes-based formula for the first 
time. The shift was phased in over three years, a transition that Tennessee officials believe was 
critical to successful implementation, so that the full impact on budget allocations was in the 
2013-14 budget request. 


The governor, legislature, and Board of Regents (which is responsible for allocations to individual 
community colleges) have all followed the outcomes-based recommendations in the THEC budget 
request when allocating higher education budgets and have indicated an ongoing commitment to 
the system. The level of funding and the consistency in application now make the state a perfect test 
case for performance funding. If the strategy works as advocates expect, it should yield strong results 
when an entire funding system is built around it. 
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become more dependent on tuition and less on appropriations as state budget cuts have taken hold. 
As Figure 1 shows, funding for outcomes in Tennessee's formula in 2014-15, especially progress 
toward degrees, balances the financial incentive of tuition. 


e Degree and certificate completion: 38 percent of state funds, 14 percent with tuition included 
e Progress toward degree and transfer: 17 percent of state funds, 6 percent with tuition included 
© Graduation rates and degrees per FTE: 7 percent of state funds, 3 percent with tuition included 


e Total degree-related funding: 63 percent of state funds, 23 percent of funds with tuition 
included 


e Total outcomes-based funding: 28 percent of funds with tuition included 


Figure 1. Proportion of Institutional Funds Allocated by Outcome Category in TN Funding Formula, Including Tuition 


100% . o ‘ 
6% % 


0.0% 
Universities Community Colleges Total 
MMM Progress & Transfer [NJ Degree & Credential [§MMJ Degree Rates [J Non-Degree-Related 
Completion Measures 


HE Tuition I) Fixed Costs & Non-Recurring Appropriations {§§§ Quality Assurance 


In addition to the degree-related measures, the formula includes measures for other important functions not directly related to degrees, such as research at 
research universities, and workforce development and dual enrollment at community colleges. These account for another 15 percent of formula funds, or 
5 percent of institutions’ total tuition and appropriations revenues. 


Tuition revenue can be seen as a performance incentive for enrollment, and it remains a significant 
component of institutional budgets. States that depend more heavily on tuition than Tennessee would 
have to allocate a larger proportion of state appropriation funds to achieve the same relative level of 
performance funding within institutions’ overall budgets. States that depend less on tuition, on the 
other hand, could allocate a smaller proportion of state funds and achieve the same level of incentive. 
The state has a policy guideline that 45 percent of four-year college funds and 33 percent of community 
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When that is factored in, the same relative level of performance funding within 
the proportion of funds in stitutions’ overall budgets. 
based on degrees and 


degree-related outcomes is: 
e Community colleges: 49 percent of state funds, 21 percent with tuition included 


e Four-year colleges and universities: 68 percent of state funds, 23 percent of funds with 
tuition included 


e In practice, the amount of actual tuition revenue institutions receive may vary from the guideline, 
but the essential point would remain that even with appropriations based on outcomes, there is 
still a substantial amount of institutional revenue not tied to the formula measures. 


What's the evidence of success? 


If performance funding is an effective strategy, states that employ it should see improvements in 
the measures used. These improvements should also exceed what would have happened without 
performance funding. If it has a big impact, the state should expect bigger changes in outcomes after 
implementation of performance funding than before. States that had not changed their funding, on 
the other hand, would not experience the same change in their outcomes trends. 


Some of the changes resulting from outcomes funding could take time to show up, both because 
the policies institutions put in place to respond to performance funding require a number of years to 
reach their full impact, and because there is a lag in data availability that would allow for comparison 
with other states that do not use performance funding. 
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Degree-award data from Tennessee 
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map perfectly to formula implementation and is not unlike trends in some states that have not imple- 
mented comprehensive outcomes funding. At the very least, it does not seem that the formula has 


damaged students’ chances for success in Tennessee, and there are limited signs that it may be helping. 


Bachelor's degrees awarded have increased by 3.4 percent annually since initial formula imple- 
mentation, compared to 2.5 percent annual growth prior to formula implementation. Other states have 
also had faster growth in bachelor's degree completion in recent years, consistent with underlying 
demographic trends, so it is too early to confidently attribute the results to the formula (Table 1) 
alone. Many other states have also implemented oo...ccccccccccccccseeeseccccececeeseessecececcceeesesseecesceceseeeseseeess 
significant higher education reforms and set 


om ; Associate degrees have increased by 
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6.3 percent annually since initial formula than the 2.8 percent average growth 
implementation, significantly faster than . . . 
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Figure 2. Associate and Bachelor’s Degree Awards at Tennessee Public Institutions 


25,000 
20,839 
20,000 18,249 
9-year CAGR: 2.5% 4-year CAGR: 3.5% 
15,000 
Enrollment-based Outcomes-based 
Funding Formula Formula 
10,000 
9,501 
5,000 7,476 


9-year CAGR: 2.8% 4-year CAGR: 6.3% 


2000-01 01-02 0203 0304 0405 05-06 0607 07-08 0809 09-10 10-11 411-12 1213 1314 
HB Associate [BI Bachelor's 


Early Results of Outcomes-Based Funding in Tennessee 4 


have also experienced significantly faster growth, partly due to rapid enrollment increases in the 
recession in 2008 and 2009 leading to higher completions two and three years afterward. The year 
immediately prior to formula adoption in Tennessee also showed strong increases in associate degree 
awards. To the extent this has anything to do with the formula, the lag time between student admission 
and degree completion means it would have to be in anticipation of the new formula, rather than 
based on actual implementation. 


Comparable national data are available for degree awards through 2012-13 and Kentucky is the 
only nearby state with easily accessible 2013-14 degree award data. Kentucky experienced 6.6 percent 
average growth in associate degrees from 2010 to 2014, roughly comparable with its rapid 6.0 percent 
growth rate prior to 2009-10 (Table 1). While it has not implemented performance funding in the 
same way as Tennessee, Kentucky has been one of the more aggressive states nationally in pursuing 
higher attainment rates for its citizens. Its policies, including large infusions of state financial aid, a 
restructured and rapidly-growing community college system, and strong state planning and account- 
ability practices—make for a challenging and imperfect comparison. The contrast, however, does 
show the risk of interpreting Tennessee trends without adequate comparative data from other states. 


Table 1. Public Institution Degree Growth Before and After Outcomes-Based Formula in Tennessee and 
Neighboring States 


Compound Average Growth Rate 
Bachelor's Degree Associate Degree* 


Post-Formula Post-Formula 
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Pre-Formula 0 2012-13 to 2013-14 
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Sources: IPEDS completion files *includes awards conferred through public 4-year institutions. 
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Those growth rates are primarily tied to a small number of institutions that ramped up certificate 
awards in direct response to the formula. One institution went from 0 to more than 500 short-term 
certificates in a single year following formula implementation. 


The “certificate” category is much more flexible than degrees, so institutions have greater scope to 
create programs very quickly or to define completion of certain existing groups of courses as a certificate 
award. In response, the Tennessee Higher Education Commission worked with institutions to refine 
standards for which certificates can be counted. The new standards will limit opportunities for "gaming" 
the formula by requiring that only certificate programs approved by the Commission may count. 


This trend can be read different ways — it is neither a clear triumph for performance funding 
nor a clear case of “abuse.” There is evidence that at least some types of credentials count more than 
simple accumulations of credits, so even if the new certificates do nothing more than define coherent 
blocks of credits that students were already earning, they could be adding value to students’ lives 
and to the Tennessee economy. On the other hand, states that include certificates in their funding 
formulas should take careful note of Tennessee's experience and ensure that the outcomes they fund 
are the ones they truly want to happen. 


Figure 3. Community Colleges Formula Outcomes Change 2009-10 to 2013-14 
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Other indicators 


Enrollment. It is not yet clear what impact Tennessee's formula will have on enrollment. On one 
hand, measures such as the graduation rate and degrees per 100 full-time-equivalent enrollments, 
actually result in more funding for enrolling fewer students, provided that degrees remain constant 
(or decline at a slower rate). On the other hand, if larger numbers of enrolled students leads to more 
progress and more completions, institutional funding will increase, and it would remain worthwhile 
for institutions to increase enrollments. Tuition is also a form of enrollment-based funding and a 
larger part of many institutional budgets than state appropriations in Tennessee. 


The net effect of these incentives might be that enrollments would grow, but degrees would grow 
faster, especially compared with other states where incentives remain primarily enrollment-based. Now 
that the state has implemented a “free tuition” policy for high school graduates attending community 
college, it will be challenging to sort out effects of that policy from those due to the funding formula. 


At four-year institutions, Tennessee's degree award growth has in fact accelerated, while enrollment 
growth rates were similar pre-and post-formula. Neighboring states (KY, GA and NC) have not seen 
the same acceleration in degree growth, and have actually had lower rates of enrollment growth than 
in pre-formula years. National data are not yet available that would allow for more comprehensive 
comparisons, so these must remain preliminary observations. 


Further, there is a legitimate concern that outcomes-based funding can provide an incentive to not 
enroll at-risk students. Properly designed funding policies acknowledge this by creating incentives 
for institutions to enroll at-risk or underrepresented students. Tennessee initially sought to address 
this issue by applying a 40 percent premium to students from certain identified focus populations— 
namely adult and low-income students. In a recent five-year review and modification of the funding 
formula Tennessee added a third student focus population—academically underprepared—and revised 
the premium for students in these groups. Based on discussion and input from institutions the 
premiums will increase—up to 120 percent—to further encourage enrollment of these students and 
recognize the additional supports necessary to ensure their success. The effects of these premiums 
on both enrollment and outcomes for the identified populations should continue to be evaluated to 
ensure the potential unintended consequences are not a result of the funding policy. 


Grades and credits earned. Both critics and proponents of outcomes-based funding might expect 
increases in student GPAs and credits completed. Proponents would expect institutions to do a better 
job advising students into the right programs and helping them finish, with the result that there 
would be fewer dropped or failed courses and higher overall grades. Critics worry about the pressure 
that outcomes-based funding places on institutions to pass students regardless of their academic 
performance. Either way, an anticipated outcome could be that a higher proportion of courses 
attempted would be passed. 


So far, that does not seem to have happened in Tennessee, though again, the available data are 
limited. The state only began collecting comprehensive student course completion information after 
implementation of the formula. Yet for the limited subset of students for whom comparable data are 
available (fall students who return the following fall), there has been no meaningful change in the 
proportion of courses completed. On the one hand, this finding suggests that outcomes funding has 
not yet led to better student performance at the course level (at least for this subset of students). On 
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of outcomes funding — it could mean there are fewer 

students in the pipeline to complete degrees in coming years. On the other hand, it could also mean 
that institutions are becoming more efficient with transfer and completion, so that fewer students in 


the pipeline are needed for the same number of degrees. Or it could be some combination of the two. 


The trend is difficult to compare with other states, given the lack of similar national data, but 
enrollments and first-time admissions have also been flattening or declining nationally, after record 
recession-era increases, largely because of the improving economy and higher numbers of college- 
age and adult residents entering or re-entering the workforce. Tennessee's latest policy innovation, 
offering free community college tuition to all high school graduates, may also increase the size of 
the population in the pipeline at these institutions. 


What to look for next 


This brief's appendix table shows how data will become available over the next several years to 
facilitate review of the impact of formula funding in Tennessee. Many Tennessee graduates have now 
spent their entire college careers at institutions funded by outcomes. Enough time has also passed for 
every institution to have adopted policies and practices better geared to the new funding incentives. 
Even so, it will take additional time for institutions to 


learn from their initial efforts to adapt to the formula and 
Many Tennessee graduates 


for the culture of higher education to continue to shift. 
The full impact of the new system will probably not be have now spent their entire 
fully known until a decade or more of systematic implemen- 


college careers at institutions 
tation has passed. 


tunded by outcomes. 
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Recommendations 


While it is too early to make firm conclusions based on Tennessee's early data alone, initial 


numbers are at least encouraging. More-specific lessons for other states (and for future evaluators of 


performance funding) that emerge from this early analysis include: 
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Establish clear baseline data as early as possible in the development of funding-formula measures 
to allow for strong, early evaluation. 


Where possible, use measures and quantities that allow for comparisons with and benchmarking 
against other states. 


Where possible, use measures that have enough history to allow for analysis of changes in 
long-term trends. 


Establish clear rules about the types of outcomes (such as the types of certificates) that will be 
included in the formula and the authority for defining or creating additional outcomes. 


Understand the full range of financial and other incentives that will exist alongside the formula 
(tuition, research funding, fundraising, executive compensation incentives) and how they are 
likely to interact. 


Do not expect short-term miracles from a long-term strategy and commit instead to several years 
of consistent outcomes-based funding to allow the approach a chance to work. 


Continue looking at measures of outcomes each year, while making well-informed comparisons 
with other states. 


In quantitative or contextual evaluations of the outcomes-based formula, make sure to account for 
the influence of other policies relevant to degree production. For instance, Tennessee recently 
created a comprehensive transfer pathway system and has modified its state financial aid system. 
These policies may also be contributing to improved degree attainment. States without outcomes- 
based funding may also be improving results because of other innovations in policy or finance. 


Understand how related measures (such as numbers of degrees and production rates) overlap 
or compete, and the extent to which they encourage similar or different institutional policies 
and practices 


Use absolute numbers of degrees rather than rates (e.g. graduation rates or degrees/FTE) as 
outcomes measures in formulas and evaluations of policy outcomes to create the clearest and 
easiest-to-measure link to increased educational attainment. 


Where to learn more 


HCM Strategists and Postsecondary Analytics conducted this review with support from Lumina 
Foundation for Education. Information about developing state performance funding systems is available 
on Lumina Foundation's Strategy Labs website at: 


http://strategylabs.luminafoundation.org/higher-education-state-policy-agenda/core-element-2/ 
adopt-and-sustain-outcomes-based-funding/ 


The Tennessee Higher Education Commission posts comprehensive information about the 
outcomes-based formula, including downloadable copies of the Excel workbooks used to generate 
allocations, on its website at: 


http://tn.gov/thec/index.html 


Additional Note: Institutions Respond to Performance Funding 


For performance funding to work, two things have to happen. First, institutions need to respond 
by improving policies and practices in different and effective ways. Second, the collective effect of 
those changes must result in serving more students, rather than just competing more intensely for 
the same pool. On the second count, data like those discussed in this brief are starting to tell the 
story. But there is no question that institutions are responding. Some examples include: 


e Austin Peay University has overhauled its advising system to ensure that students stay on track 
to degree completion. See the New York Times infographic on the system at: 
http://www. nytimes.com/interactive/20 12/07/18/education/edlife/ 
student-advising-by-algorithm.html?_r=0 


e University of Tennessee at Knoxville changed to block tuition policy so students pay the 
same to take 15 or 18 credits as they would pay for 12. See: 
http://web.utk.edu/~ bursar/Fall2012FeesUG. pdf 


© Pellissippi State Community College has greatly expanded its certificate offerings, 
providing short-term credentials for both full-time students and working adults. See: 
http://www. pstcc.edu/academics/certificates. php#. UW njeLXU-Sr 
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Appendix: Availability of Data to Support Quantitative Comparisons of Tennessee Outcomes with Other States 


IPEDS Data Availability 


Current 
Academic 
Year 


2008-09 
2009-10 
2010-11 
2011-12 
2012-13 
2013-14 
2014-15 
2015-16 
2016-17 
2017-18 
2018-19 
2019-20 
2020-21 


Outcomes-Based Formula 
(OBF) Introduced 


Budget 
Requested 


Fall 2008 
Fall 2009 
Fall 2010 
Fall 2011 
Fall 2012 
Fall 2013 
Fall 2014 
Fall 2015 
Fall 2016 
Fall 2017 
Fall 2018 
Fall 2019 
Fall 2020 


Budget 
Requested 
For 


2009-10 
2010-11 
2011-12 
2012-13 
2013-14 
2014-15 
2015-16 
2016-17 
2017-18 
2018-19 
2019-20 
2020-21 
2021-22 


Budget 
Formula 
Used 


Enrollment 
Based 


OBF 
Full 
Implemen- 
tation 


Enrollment 


Fall 2007 
Fall 2008 
Fall 2009 
Fall 2010 
Fall 2011 
2012 
2013 


Retention 
Rate 


F06 Cohort 
FO7 Cohort 
F08 Cohort 
FO9 Cohort 
F10 Cohort 
F11 Cohort 
F12 Cohort 
F13 Cohort 
F14 Cohort 
F15 Cohort 
F16 Cohort 
F17 Cohort 
F18 Cohort 


Early Results of Outcomes-Based Funding in Tennessee 


Completion 


2006-07 
2007-08 
2008-09 
2009-10 
2010-11 
2011-12 
2012-13 
2013-14 
2014-15 
2015-16 
2016-17 
2017-18 
2018-19 


Graduation Rate 


3-Year 6-Year 


F04 Cohort 
FO5 Cohort 
F06 Cohort 
FO7 Cohort 
F08 Cohort 
FO9 Cohort 
F10 Cohort 
FI] Cohort 
F12 Cohort 
F13 Cohort 
F14 Cohort 
F15 Cohort 
F16 Cohort 


FOI Cohort 
FO2 Cohort | Fa 
F03 Cohort 
F04 Cohort 
FO5 Cohort 
F06 Cohort 
FO7 Cohort 
F08 Cohort 
FO9 Cohort 
F10 Cohort 


Adult 
tudent 


S 
Enrollment 


N/A 
2008 


Pell Students 


All Students | Freshmen 
N/A — | F06 Cohort 
2007-08 | FO7 Cohort 
2008-09 | F08 Cohort 
2009-10 | FO9 Cohort 
2010-11 | F10 Cohort 
2011-12 | F11 Cohort 
2012-13 | F12 Cohort 
13-14 | F13 Cohort 
14-15. | F14 Cohort 
15-16 | F15 Cohort 
16-17. | F16 Cohort 
17-18 | F17 Cohort 
18-19 | F18 Cohort 


Where We Are Now 
(March 2015) 


